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Generalized Polya urn models can describe the dynamics of fi- 
nite populations of interacting genotypes. Three basic questions these 
models can address are: Under what conditions does a population 
exhibit growth? On the event of growth, at what rate does the pop- 
ulation increase? What is the long-term behavior of the distribution 
of genotypes? To address these questions, we associate a mean limit 
ordinary differential equation (ODE) with the urn model. Previously, 
it has been shown that on the event of population growth, the limit- 
ing distribution of genotypes is a connected internally chain recurrent 
set for the mean limit ODE. To determine when growth and conver- 
gence occurs with positive probability, we prove two results. First, if 
the mean limit ODE has an "attainable" attractor at which growth is 
expected, then growth and convergence toward this attractor occurs 
with positive probability. Second, the population distribution almost 
surely does not converge to sets where growth is not expected and al- 
most surely does not converge to "nondegenerate" unstable equilibria 
or periodic orbits of the mean limit ODE. Applications to stochastic 
analogs of the replicator equations and fertility-selection equations of 
population genetics are given. 

1. Introduction. The founder-effect in population genetics refers to the 
establishment of a new population consisting of a few founders that is 
isolated from the original population. A founder-effect can occur when a 
small number of individuals colonize a place previously uninhabited by their 
species. In this case, the founding population is geographically isolated from 
the original population. A founder-effect due to temporal isolation can oc- 
cur when a population passes through a bottle neck after which only a few 
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individuals survive. Several fundamental questions surrounding the founder- 
effect include: What is the probability that a founding population success- 
fully establishes itself? If a founding population establishes itself, what is the 
population's growth rate and what is the long-term genotypic or phenotypic 
composition of the population? How does the initial genotypic composition 
and initial population size influence the likelihoods of the various outcomes? 

To address these questions, we consider a general class of Polya urn models 
introduced in [12]. Traditionally, Polya urn models are described as involving 
an urn, which contains a finite number of balls of different colors. At discrete 
moments in time, balls are added or removed from the urn according to 
probabilities that only depend on their distribution and number at that 
point in time. The pertinence of these models to evolutionary questions is 
self-evident if we view the balls as individuals whose color represents their 
genotype or phenotype, adding or removing balls as replication or death 
of individuals and updates of the urn as interactions between individuals. 
When balls are added at a constant rate (i.e., a fixed number of individuals 
are added at every update of the process), Polya urn models have been 
studied extensively by Arthur, Ermol'ev and Kaniovskii [1], Benaim and 
Hirsch [5, 6], Hill, Lane and Sudderth [8], Pemantle [10] and Posch [11] 
amongst others. What makes the models introduced in [12] more relevant 
to population processes is that they permit the removal of balls, as well as 
the addition of balls at nonconstant rates. Consequently, extinction of the 
entire population or one or more subpopulations is possible in finite time. 

The article is structured as follows. In Section 2 we introduce the class 
of generalized urn models. As examples, replicator processes and fertility- 
selection processes with and without mutations are introduced. These urn 
models typically predict that either the populations go extinct or grow, and 
that demographic stochasticity is most pronounced when the population 
is small. Once the population starts to get large, it tends to grow in an 
essentially deterministic fashion. For this reason, on the event of nonextinc- 
tion, the dynamics of the distribution of types in the population (i.e., the 
distribution of the color of balls in the urn) are strongly correlated to the 
asymptotic behavior of an appropriately chosen ordinary differential equa- 
tion (ODE), commonly called the mean limit ODE. In Section 3 we describe 
the mean limit ODE, and recall a theorem [12] that on the event of growth 
relates the asymptotic behaviors of the stochastic process to its mean limit 
ODE. Using this result, we derive a time averaging principle and a compet- 
itive exclusion principle for replicator processes. We also show that additive 
fertility-selection processes almost surely converge on the event of nonex- 
tinction to a fixed point of the mean limit ODE. These results, however, 
provide no insight into when population growth occurs with positive proba- 
bility and which limiting behaviors occur with positive probability. In [12], 
Theorem 2.6, it was shown that if the population grows with probability 1 
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(i.e., more balls are being added than removed at each update), then the 
population distribution converges with positive probability to "attainable" 
attractors for the mean limit ODE. However, for most evolutionary and 
ecological processes extinction occurs with positive probability and, conse- 
quently, the almost surely growth assumption is not meet. In Section 4 we 
remedy this issue and prove that if there is an "attainable" attractor for the 
mean limit ODE at which growth is expected (i.e., on average more balls are 
added than removed), then there is a positive (typically less than 1) proba- 
bility that the population grows and the population distribution converges 
to the attractor. In addition, we provide an estimate for this probability and 
for the rate of growth. In Section 5 we prove nonconvergence to invariant 
subsets of the mean limit ODE where growth is not expected and noncon- 
vergence to "nondegenerate" unstable equilibria and periodic orbits of the 
mean limit ODE. In Section 6 we combine our results to give necessary 
and sufficient conditions for growth with positive probability for processes 
with gradient-like mean limit ODEs and apply these conditions to additive 
fertility processes with mutation. 

2. Generalized urn models. In this section we introduce a class of gener- 
alized urn models. This class generalizes the urn models introduced in [12]. 
Three examples or evolutionary processes described by these generalized 
urn models are given in the sections below. One of these processes, replica- 
tor processes, was described previously in [12] and stochastic versions of the 
replicator equations [9]. The other two processes, fertility-selection processes 
with and without mutation, are stochastic versions of the fertility-selection 
equations with and without mutation [9]. 

Due to the fact that we are dealing with finite populations consisting 
of individuals that are one of k types, we consider Markov chains on the 
positive cone 



We shall always write || • || for the Euclidean norm on M. k . 

Let z n = (z n , ■ • ■ , z k ) be a homogeneous Markov chain with state space . 
In our context, z % n corresponds to the number of balls of color i at the nth 
update. Associated with z n is the random process x n defined by 



Z k + = {z = (z 1 , . . . , z k ) E Z k : z i > for all i] 
of the set Z fc of /c-tuples of integers. Given a vector w = (w 
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which is the distribution of balls at the nth update. Note that when there 
are no balls at the nth update, we set x n to zero which we view as the "null" 
distribution. Let Sk C M fc denote the unit k — 1 simplex, that is, 



Let II : x i— ► [0, 1] denote the transition kernel of the Markov chain z n . 
In other words, 11(2, z') = P[z n+ \ = z'\z n = z\. We place the following as- 
sumptions on the Markov chains z n of interest: 

(Al) At each update, there is a maximal number of balls that can be added 
or removed. In other words, there exists a positive integer m such that 
\z n+ \ — z n \ <m for all n. 

(A2) There exist Lipschitz maps 



Since we view updates of the Markov chain to correspond to the effect 
of interactions between individuals, assumption (Al) implies that each in- 
teraction results in the addition or removal of no more than a maximum 
number of individuals. Assumption (A2) assures that there is a well-defined 
mean limit ODE for the urn models. 

2.1. Replicator processes. Consider a system consisting of a finite pop- 
ulation of individuals playing k different strategies. At each update of the 
population, pairs of individuals are chosen randomly with replacement from 
the population. The chosen individuals replicate and die according to prob- 
abilities that only depend on their strategies. More precisely, let m be a 
nonnegative integer that represents the maximum number of progeny that 
any individual can produce in one update. Let {R n }n>o and {i? n }n>o be se- 
quences of independent identically distributed random k x k matrices whose 
entries take values in the set {— 1,0, 1, . . . , m}. Let R l J and R%{ denote the 
ijth entry of R n and R n , respectively. Let {r n } n >o be a sequence of inde- 
pendent identically distributed random k x 1 matrices whose entries take 
values in the set {—1, 0,1,..., m}. We define a replicator process according 
to the following rules: 

1. Two individuals are chosen at random with replacement from the the 
population. Make note of the individuals chosen and return them to the 
population. 




x = (x 



i 




{Pw :S k ^[0,l]:we Z fc , \w\ < m) 



and a real number a > 0, such that 

\p w (z/\z\) - U(z,z + w)\ < a/\z 
for all nonzero z £ TLh and w G Z fc with \w\ <m. 
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2. If the same individual is chosen twice and it plays strategy i, then r n 
individuals of strategy i are added to population. 

3. If two distinct individuals are chosen, say strategy i and strategy j, then 
add individuals of strategy i and add R^ individuals of strategy j. 

It is not difficult to verify that this process satisfies assumptions (Al) and 
(A2) with p w (x) = xVPfi^ + Rf = w*] + V^.^.r'.r' /' /,"/ = w^P^ = 
0], whenever w = (0, . . . ,0, w % , 0, . . . , 0) with w l ^ 0, p w (x) = 2x % x^P\R( = 
w l ]P[Rf = wi], whenever w = (0, . . . ,0,w l ,0, . . . ,0,u> J ,0, ... ,0) with w 1 ^ 0, 
w 3 ^ 0, and i ^ j, p w (x) = 0, whenever w has three or more nonzero coordi- 
nates, and p (x) = 1 - J2 w ^oPw( x )- 

2.2. Fertility-selection processes. Consider a population of diploid indi- 
viduals that has k distinct alleles Ai,...,Ak that occupy a single locus. 
Assume that the population is monoecious (i.e., there is only one "sex") 
and that each individual chooses its mate randomly from the population. 
We assume that individuals die immediately after mating. Although there is 
no distinction between individuals of genotype AiAj and AjAi, we develop 
an urn model of the form z n = (z l J) G Z^_ xfc as it is notationally more conve- 
nient. For i ^ j, let z 1 ^ = zj^ denote the number of individuals of genotype 
AiAj at the nth update of the population. Alternatively, let z% denote twice 
the number of individuals of genotype A{Ai at the nth update of the popu- 
lation. Hence, \z n \ = J2i,j z n equals twice the total number of individuals in 
the population. 

For every pair of genotypes, say AiAj and A r A s , we associate a sequence of 
i.i.d. random variables G n (ij, rs) = G n (rs, ij) that take values in {0, ... , m}, 
where m represents the maximal number progeny produced by a mating and 
where G n (ij,rs) represents the number of progeny produced by a mating 
between genotypes AiAj and A r A s at update n. Let z n 6 Z^. xfc be a Markov 
chain satisfying z£ = z-£ and updated according to the following rules: 

1. If the population size is less than two, then the population goes extinct. 
In other words, if \z n \ < 4, then z n +i = 0. 

2. Pick two individuals at random without replacement from the population, 
say genotypes AiAj and A T A a . 

3. Remove the chosen individuals from the population (i.e., they die). 

4. Add G n (ij,rs) individuals to the population. The genotype, say A U A V , 
of each added individual is independently determined by random mating 
probabilities (i.e., u equals i or j with equal probability and v equals r 
or s with equal probability). 

Define 

\%n I 

0, if z n = 0. 



G 
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Hence, if i ^ j, then 2x 13 = 2x 3% equals the proportion of the population with 
genotype A^Aj. Alternatively, x u is the proportion of the population with 
genotype A^A^ 

To see that this process satisfies assumption (A2), let w and z be in Z+ xfc 
such that w %3 = w 3% and z %3 = z Jl for all 1 < i,j < k. Assume that \z\ > 0. 
Pw( z /\ z \) is given by a linear combination of the terms x l3 x TS . On the other 
hand, Il(z, z + w) is given by the corresponding linear combination of the 
terms x 4:, x' rs |z|/(|z| —2), when {i, j} ^ {r, s} and x l3 {x rs \z\ —2)/{\z\ —2). 
From these observations it follows that assumption (A2) is satisfied. 

2.3. Fertility- selection process with mutations. To account for mutations 
in the fertility-selection process, let (J,(ij,rs) > for 1 < i,j,r,s < k be such 
that J2r<s rs ) = 1 for all 1 < i < j < k. The quantity fj,(ij, rs) represents 
the probability that the genotype AiAj mutates to the genotype A r A s . The 
fertility process with mutation is given by the first three rules of the fertility 
process without mutation and replacing the fourth rule with: Add G n (ij,rs) 
individuals to the population. For each added individual, its genotype is 
determined by two steps. First, determine a genotype A U A V according to 
random mating probabilities. The probability that the added individual has 
genotype AqAy is given by fj,(uv,uv). For reasons similar to the fertility- 
selection process without mutation, this process also satisfies assumptions 
(Al) and (A2). 

3. Mean limit ODEs. To understand the limiting behavior of the x n , we 
express x n as a stochastic algorithm using the following lemma. 

Lemma 1. Let z n be a Markov chain on satisfying assumptions (Al) 
and (A2) with mean limit transition probabilities p w :Sk — > [0,1]. Let T n 
denote the a field generated by {zq, z±, . . . , z n } . There exists sequences of 
random variables {U n } and {b n } adapted to T n , and a real number K > 
such that: 

(i) if z n ^0, then 

(1) x n+ i -x n = — [ ^2 Pw{x n )(w - x n a(w)) + U n+ i + b n+1 J , 

(ii) E[U n+1 \z n } = 0, 

(iii) \\U n \\ <4m and £[||f7 n+ i|| 2 |.Fn] <4m 2 , 

(iv) ||&„ + l|| < maxjfj^l)- 

Proof. The proof of this lemma is very similar to the proof of Lemma 
2.1 in [12]. Consequently, we only provide an outline of the proof. Define 

9( x ) = Pw(x)(w-xa(w)), 
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Un+1 — {%n+l %n E[x n +i \^n\ ) \%n | j 

b n +i = \z n \E[x n+ i - x n \z n ] - g(x n ). 

From these definitions it follows that (i) and (ii) hold. For the remainder of 
the proof, let z = z n and x = x n . To prove (hi), notice that if z n+ \ ^ 0, then 
it can be shown that 



(2) 



\{x n+ i - x)\z\\\ < 2m, 



as no more than m balls are being added or removed at any update. Al- 
ternatively, if z n +i = 0, then it must be that \z n \ < m since no more than 
m balls can be removed at a single update. In which case, x n+ \ = and 
||(x n+ i — x n )|z n ||| = \\z n \\ < fn. From this (ii) follows. To prove (iv), notice 
that if z 7^ 0, then 



(3) 



\z\E[x 



n+l 

E 

Wy^ — Z 



\z\(w — a{w)x) 



\z\ + a(w) 
If \z\ > m, (3) can be used to show 



U(z,z + w) - zU(z,0). 



3 n.+l 



< 



E 

\w\<m 



a(w) 



LI(z, z + w) — p w (x)^j (w — a(w)x) 



Applying assumptions (Al) and (A2) implies that there is K\ > such that 
||&n+i|| ^ K\/\z n \, whenever \z n \ > m. On the other hand, if \z\ < m, then 
the definition of b n+ i implies that ||6 n +i|| < 2m + sup xgSn Choosing 
K sufficiently larger than K\ completes the proof of (iv). □ 



The recurrence relationship (1) can be viewed as a "noisy" Cauchy-Euler 
approximation scheme with step size \j\z n \ for solving the ordinary differ- 
ential equation 

(4) ^= E Pw{x)(w - xa(w)), 

which we call the mean limit ODE. When the number of individuals in 
the population grow without bound, the step size decreases to zero and it 
seems reasonable that there is a strong relationship between the limiting 
behavior of the mean limit ODE and the distribution of balls x n . To make 
the relationship between the stochastic process x n and the mean limit ODE 
more transparent, it is useful to define a continuous time version of x n 
where time is scaled in an appropriate manner. Since the number of events 
(updates) that occur in a given time interval is likely to be proportional to 
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the size of the population, we define the time r n that has elapsed by update 
n as 



To relate the limiting behavior of the flow cfit(%) of (4) to the limiting 
behavior of Xt, Schreiber [12] proved the next theorem using techniques of 
Bena'im [2, 4]. Recall, a set C is called invariant for the flow (fit provided 
that (j>t{C) = C for all t € M.. A compact invariant set C is internally chain 
recurrent provided that for every i£C, T > and e > 0, there exist points 
xi,X2 ■ ■ ■ ,x s in C and times t\,...,t s greater than T such that x\ = x s = x 
and \\4)ti{xi) — < £ for 1 < i < s — 1. Given a function Xt :M+ — > M. k or 

a sequence {x n } n >o in we define the limit sets, L(Xt) and L(x n ), of 
Xt and x n . L(Xt) is the set of p € M fc such that lim/^oo Xt k =p for some 
subsequence {tk}k>o with lim^ooifc = oo. L(x n ) is the set of p G M fc such 
that limfc_ >00 x nfe =p for some subsequence {nk}k>o with limfc^oon^ = oo. 

Theorem 1 ([12]). Let z n be a Markov process satisfying assumptions 
(Al) and (A2) mean limit ODE (4). T/ien on the event {liminfj^oo > 



1. The interpolated process Xt is almost surely an asymptotic pseudotrajec- 
tory for the flow (fit of the mean limit ODE. In other words, Xt almost 
surely satisfies 



for any T > 0. 

2. The limit set L(Xt) of Xt is almost surely an internally chain recurrent 
set for the mean limit ODE. 

The first assertion of the theorem roughly states that Xt tracks the flow of 
the mean limit ODE with increasing accuracy far into the future. The second 
assertion of the theorem states that the only candidates for limit sets of the 
process x n corresponding to the distribution of balls are connected compact 
internally chain recurrent sets for the mean limit flow. 

To give a sense of the utility of this result, we derive some corollaries for 
the replicator processes and the fertility-selection process in the next two 
sections. 



r = 




The continuous time version of x n is given by 




0}: 



lim sup 1 1 <fi h X t - X t+h ||=0 

*^°°0<h<T 
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3.1. Implications for replicator processes. Let {i? n }n>o and {i? n }n>o be 
sequences of independent identically distributed random k x k matrices 
whose entries take values in the set {—1,0, 1, . . . , m}. Let {r n } n >o be a se- 
quence of independent identically distributed random k x 1 matrices whose 
entries take values in the set { — 1,0, 1, . . . ,m}. Let z n £ Z+ be the replica- 
tor process associated with these random matrices. Define the mean payoff 
matrix by A = E[Rq + R$\. The limiting mean ODE associated with this 
process is given by a replicator equation [9] 

olx 

(6) — = di&g(x)Ax — (x.Ax)x, i = l,...,k, 

at 

where x.A denotes multiplying the left-hand side of A by the transpose 
of x and diag(x) is a diagonal k x k matrix with diagonal entries x l . The 
dynamics of (6) are well studied and have two remarkable properties whose 
proofs can be found in [9]. 

Theorem 2 (Exclusion principle). If the replicator equation (6) has no 
equilibrium in intSfc, then every orbit of (6) converges to dS^- 

Theorem 3 (Time averaging principle). If the replicator equation (6) 
has a unique equilibrium p in intiS& and if x(t) is a solution of (6) such that 
L(x(t)) C int Sk, then 

1 f T 

lim — / x(t) dt = p. 
T^oo T Jo W 

It turns out that Theorem 1 provides us the tool in which to transfer 
these theorems to replicator processes. 

Theorem 4. Let z n be a replicator process on Z* with mean payoff 
matrix A. If mean limit replicator equation has no equilibria in intS^, then 
L{x n ) n dSk 7^ almost surely on the event {liminfn^oo ^ > 0}. 

Proof. The proof of Theorem 2 implies there exists a vector c G M fc such 
that the function V(x) = T^c*logx* is strictly increasing along the forward 
orbits of the mean limit replicator equation that lie in int . Consequently, 
every compact connected internally chain recurrent set intersects dS/.. Ap- 
plying Theorem 1 completes the proof. □ 

Theorem 5. Let z n be a replicator process on with mean payoff ma- 
trix A. Let Xt be continuous-time process associated with z n that is defined 
by (5). Assume that (6) has a unique rest point p in intS^. Then 

1 f T 
lim — / Xfdt = n 
T^oo T Jo 
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almost surely on the event 



(liminf i^i > ol n \L(X t ) C mtS k \. 

[_ n.->oo n ) 



Proof. Consider a trajectory Xt from the event {lim inf n ^oo -^zr > 
0} n {L(Xt) C int Sk}- Theorem 1 implies that Xt is almost surely an asymp- 
totic pseudotrajectory for the flow (fit of the mean limit replicator equation. 
Theorem 1 implies that L(Xt) is a compact internally chain recurrent set 
for the flow (fit of (6). Theorem 3 implies that 

1 [* 

lim — / (fi s x ds = p 

t^co t Jo 

for all x 6 L(Xt), and this convergence is uniform. Therefore given e > 0, we 
can choose T > and a compact neighborhood U of L(Xt) such that 



T 







s xds — p 



£ 

<3' 



whenever x € ?7. Since X t is an asymptotic pseudotrajectory, there exists 
an I > 1 such that 

e 

sup ||Af +h - (fi h Xt\\ < - 

0<h<T o 



for all t>lT. For any % £ Z + , define 







s X iT ) ds 



+ 



/ (^ s Xj T - p) ds 
Jo 



Since L(Xt) C U, there is an N > I such that Xt £U for all t > iVT. Given 
any tGl, let [t] denote the integer part of t. For any t > (N + 1)T, we get 

1 /•* 

- / X s ds-p 
t Jo 



NT 



(X s -p) ds 



+ ^(iY)+V(iV + l) + 

■t 

[t/T]Q 



1 + 



f (X s -p)ds) 

J\t/T]T J 



N)+2 t 



T 



Taking the limit as t —> oo, we get that 



lim sup 

t— >oo 



1 ft 

- X s ds-p 
t Jo 



Taking the limit as e — > completes the proof of the theorem. □ 
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3.2. Implications for additive fertility-selection processes. Let z n £ Z+ xfc 
be a fertility-selection process defined by the sequence of random variables 
G n (ij,rs) with 1 < i,j,r,s < k. Define g(ij,rs) = E[Go(ij,rs)]. Define x n = 
z n /\z n \, whenever z n ^ and x n = otherwise. 

The mean limit ODE for this selection-fertility process is given by the 
fertility-selection equations (see, e.g., [9]) 

dx ij A 

(7) — - = }2 9(ir,js)x tr x JS - x lJ g, 



dt 

r,s=l 



where 



g= g{ir,js)x %r x :,s . 

Now consider the special case, when each allele contributes additively to 
number of progeny produced by a mating. In this case, if jij is genotype 
AiAj's contribution to fertility, then the mating between genotypes AiAj 
and A r A s produces on average 

g(ij,rs) =7y'+7rs 
progeny. Under this additional assumption, equation (7) simplifies to 

dx ij 

(8) — = x3 li + x l lj -2x l3 *(, 

where x % = J2j=i x ^ ls ^lie frequency of the allele Ai in the population, 

k 

r=l 

is the average fertility of allele i in the population and 

k k 

i=l *J=1 

is the average fertility of the population. 



Theorem 6. If z n is an additive fertility- selection process and the 
mean-limit ODE (8) has only a finite number of equilibria, then on the event 
{liminf^^oo > 0}, x n almost surely converges to an equilibrium of (8). 

The proof of this theorem follows from the work of Hofbauer and Sig- 
mund [9] that we include here for the reader's convenience. 
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Proof of Theorem 6. Define the Hardy- Weinberg manifold by 
H ={x: x lj = x % x? for all 1 < i,j < k}. 
Since for any solution x(t) to (8) 

j t {x ij {t) -x\t)x\t)) = -(xV(t)-x\t)xi(t))2j, 

x tJ (t) — x t (t)x :, (t) converges exponentially to zero. Hence, all compact con- 
nected internally chain recurrent sets lie in the Hardy- Weinberg manifold. 
On the Hardy- Weinberg manifold, the dynamics of (8) are determined by 
the Hardy- Weinberg relations x %3 = x l x 3 and the differential equation 



This differential equation is the continuous-time selection equations with 
selection parameters 7« and, consequently, the mean fertility 7 is a strict 
Lyapunov function for (9) (see, e.g., [9]). Hence, all compact connected inter- 
nally chain recurrent sets correspond to compact connected sets of equilibria. 
Since we have assumed that there are only a finite number of equilibria, the 
only compact connected internally chain recurrent sets are individual equi- 
libria. Applying Theorem 1 completes the proof of this theorem. □ 

4. Growth and convergence with positive probability. Theorem 1 helps 
to determine the limiting behavior of the genotypic composition of a popu- 
lation on the event of growth. However, it does not indicate which limiting 
behaviors occur with positive probability and sheds no insight into condi- 
tions that ensure that the population grows with positive probability. The 
goal of this section is to show that when the mean limit ODE admits an 
attractor at which growth is expected, the population grows with positive 
probability and its genotypic composition converges to the attractor with 
positive probability. Prior to stating and proving this result, we prove the 
following proposition that estimates the rate of growth on the event of con- 
vergence to a set where growth is expected. 

Proposition 1. Let z n be a generalized urn process satisfying assump- 
tions (Al) and (A2). Let K C St be a compact set. If 



(9) 



dx l 
lit 



7i - X% 1- 



A= inf } p w (x)a(w) >0 



w 



then 



lim inf > A 



n — >oo n 



on the event {L({x n }) n >o C K} n {linin^oo \z. 



■n 



OO}. 
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Remark. If K is an equilibrium, liminL^oo = A on the event {L({x n }) n >o C 
K} n {lim^oo \z n \ = oo}. 

Proof of Proposition 1. Let 

£ = {L({x n }) n>0 CK}n\ lim \z n \ = oo\. 

We will show that liminfn^oo \z n \/n > A — e on the event £ for every e > 
0. Let e > be given. The definition of A, compactness of K, continuity 
of p w and assumption (A2) imply that there exist an integer / and com- 
pact neighborhood U of K such that £^[|z n +i| — \z n \\z n = z] > A — e, when- 
ever \z\ > I and z/\z\ £ U. For each natural number j, define the event 
£j = {\ z n\ > I, z n/\z n \ £U for n > j}. Notice that £ C Ufci £j- Define a se- 
quence of random variables by Nq = and 



N n+1 
for n > 0. Let 



\z n+l \ - \z n \, if \z n \ > I and z n /\z n \ € U, 
A, otherwise, 



n 1 



M n = T,-(N i -E[N i \r i .. 1 \). 
M n is a martingale that satisfies 



sup£[M2]<4m 2 ]r4> 

i>i 2 

as \Ni\ < 2m. Therefore, by Doob's convergence theorem {M n } n >i converges 
almost surely. By Kronecker's lemma, 

1 n 

(10) lim - J2Ni-E[Ni\Ji.x] = 

i=l 

almost surely. Since Ya=j+i Ni = \z n \ — \zj\ for all n > j on the event £j and 

E[Ni\Fi-i] > A - e for all i > 1, (10) implies that lim inf > A - e 

almost surely on the event £j . It follows that lim inf n ^oo > A — e almost 
surely on the event £. □ 

Let 0t(x) denote the flow of the mean limit ODE in (4). A compact 
invariant set A C S/. is called an attractor provided that there is an open 
neighborhood U C Sfc of A such that 



f)\J^ s U = A. 

t>0s>t 
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The basin of attraction B(A) of A is the set of points x G Sk satisfying 
infyg^ \\4>tx — y\\ — > as t — ► oo. 

Define the set of attainable points, Attoo(X), as the set of points x € Sk 
such that, for all M £ N and every open neighborhood C/ of x 

-P[kn| > M and x n for some n] > 0. 



Theorem 7. Lei z n £>e a generalized urn process satisfying assumptions 
(Al) and (A2). Let A be an attractor for mean limit ODE with basin of at- 
traction B(A). Assume that 

A = inf p w (x)a(w) > 

w 

and define 

C = (liminf ^ > A j n {L({x n } n>0 ) C A}. 

L n->oo raj 

If U is an open set, whose closure is contained in B{A), then there exists 
a constant K > suc/i i/icrf /or a/Z M£N, 

P[C] > (l — rrl^'O' 2 "! ^ M and x n £U for some n]. 



MJ 

In particular, if 

B{A)C\Ktt oo {X)^0 

then P[C] > 0. 



Remark. Theorem 7 simultaneously provides a condition that ensures 
that the population grows with positive probability and that the distribu- 
tion of the population converges to an "attainable" attractor. Consequently, 
this result significantly improves ([12], Theorem 2.6) that proved conver- 
gence with positive probability to "attainable" attractor under the strong 
assumption of population growth with probability one. 

Proof of Theorem 7. Assumption mi x& ^^2 w p w {x)a{w) >0 means 
that the population grows in a neighborhood of the attractor A when the 
population size is sufficiently large. It implies that there exist a\, 02 > and 
a neighborhood J\f of A such that £[|z n+ i| - \z n \\z n ] > ail{ Xn eAr,|2 n |>a 2 }- 
The proof relies on the following principle: remaining in a neighborhood of 
the attractor increases the population size and this increase in population 
size increases the likelihood of remaining near the attractor. Let U be an 
open set such that U is a compact subset of B{A). Assume M S N and 
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p G N are such that > M, x p G U] > 0. Choose a neighborhood V of A 

such that is a compact subset of J\f n B(A). Since A is an attractor there 
exists a time To > 1 such that the trajectories coming from U UV rejoin the 
neighborhood V after time To. More precisely, there exists 5 > such that, 
if X t G 17 U V, T > T and ||<M*t) ~ x t+r\\ < 6, then X t+T G V. 

To avoid double subscripts, we let z(r) denote z r , r(r) denote r r , etc. 
Define r$ = p and 

r fc = inf {r > r fc _i,r(r) - T(r k -i) > T } 

for all k > 1. Since r„+i — r„ > i — 4 — for all n where m is the integer in 

assumption (Al), lim n _+ 00 r n = oo and r n < +oo for all n. Define A = e T ° m , 
B = 3 A and the following events for all k > 1: 

Ei(fc): ^(r^l^C^^M, 

^(fc): for all r G [r fc , r fc+ i], x r G V, 
where ( = l + aiT /2B. Let E(0) be the event {|z p | > M, x p G f7}. For fe > 1, 
define -E'(fc) = E(k — 1) D Ei(k) H E2(k) . We will show that there exists a con- 
stant F > such that P[J5(fc + 1)|T(0)] > P[E(k)\E(0)] - F/MQ k for all 
k > 0. The proof of this estimate relies on three lemmas. The first consists 
in observing that the population size \z r \ remains bounded on time inter- 
vals of order To, namely between 5 _1 |z(rfc)| and B\z(rk)\ on [rfc, rfc+i]. The 
second one makes use of this claim to underestimate probability of being 
inside V on [r k+ i , r k j r 2\ if x ( r k) &U UV. The third lemma estimates the 
probability that the population grows sufficiently. 

Lemma 2. For enough large \z(r k )\: 

1. B\z(r k )\ > \z r \ > 5 _1 |2:(r fc )| for all r G [r k ,r k+1 ], 

2. T <r(r k+1 )-T{r k )<2T , 

3. T S _1 |2;(r fe )| < r k+1 - r k <T B\z(r k )\. 

PROOF. Suppose \z(r k )\ > Am. Let u = inf{n G N: \z(r k + n)\ < A~ l \z{r k )\}. 
Then 

r(r k + u) -r(r k ) > ^ 



> 



0^<|-^)|(l-A-l)/m A " 1 l ar ( r *)l + ^ 
^(rjbJKl-A-^/m 



o .A 1 |z(rfc)| + mx 

= -\n(A)=T . 

m 

This proves that A _1 |2(rfc)| < \z(r)\ for all r G [?"fc,rfc+i), which implies 
for sufficiently large |z(rfc)| that i?~ 1 |z(rfc)| < \z(r)\ for all r G 
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One can show similarly that 274|z(rfc)| > \z(r)\ for all r € [r^, r^+i] . The def- 
inition of r k immediately implies that r(rfc+i) — T(r k ) > To. Since To > 1 
and r(n + 1) — r(n) < 1 for all n, the definition of also implies that 
r(rfc +1 ) — r(rfc) < To + 1 < 2Tq. The proof of claim 1 and the fact that To > 1 
imply that pg^ffh > r(r k+1 ) - r(r k ) > T and < r(r) - r(r fc ) < T 

for all r € [r^r^i). Claim 3 follows for sufficiently large |z(rfc)|. □ 

Lemma 3. There exists a C > depending only on p w , To , 5, a\ and 
m such that 

P[^(fc + l)|£;i(A;),x(r fc )€?7uy]>l 
/or all k>0 and M > sufficiently large. 



Proof. Suppose E\{k) is satisfied and x(r&) £UUV. Lemma 2 implies 
that for large enough M, \z(r)\ > B~ 2 \z(r k )\ for r G [r k ,r k+ 2], 4To > r(r) — 
r(r k ) > T for r £ [r fe+1 ,r fc+2 ] and r fc+2 - r k < 2B 2 T \z(r k )\. Define 

5(0;) = ^2p w (x)(w - xa{w)). 



Let L be the Lipschitz constant for g and ||<?||o = sup \\g(x) \\ . Using Gron- 
wall's inequality, we prove the following estimate: 

(11) sup \\4>r(r)-r(r k )x(.r k ) - x(r)\\ < e 4iT ° (Ti (r k , r k+2 ) + T 2 (r fe , r k+2 ) ) , 



where 



T 1 (r k ,r k . 



sup 

» , fc<i<rfe+2-l 



tf(t + l) 



1*01 



and 



r 2 (r fc ,r fc+2 ) 



2 sup ||p(a;) 



inf 



r fe <r<r fe+2 



|z(r)| 



+ 



sup 



E 



b(i + l) 



1*01 



To prove (11), let X(t) = Xt denote the continuous time version of x n defined 
in (5) and c(t) = supjra 6 Z + : t > r(n)}. Notice that for any h > and i > 0, 

X(t + h) -X(t) 

c(t+h)-l 

= x(c(t + h)) - x(c(t)) = x(i + l)-x(i) 

i=c(t) 



c(t+h)-l 

E 

i=c(t) 



g(x(i))+U(i + l) + b(i + l) 

k(01 
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(c(t)) 

r(c(t+h))-t 
r(c(t))-t 



g(X(s))ds + 

i=c(t) 



U(i + l)+b(i + l) 



z li 



c(t+ft.)-l 



5 (X(t + s))ds+ ]T 



Ef(i + 1) + 6(i + 1) 



i=c(t) 



Since fc A"(t) = + J ft g(</> 8 X(t)) ds, the previous equalities imply that 



& ft (X(t))-X(t + fc)|| 



< / \\g(MX(t)))-g(X(t + s))\\ds^ 
\\ g (X(t + S ))\\ds + 



+ 



r(c(t+h))-t 



r(c(t))-t 
c(t+h)-l 

E 

j=c(t) 



\\g(X(t + s))\\ds 
U(i + l) + b(i + l) 



rh 

<L \\<p s (X{t))-X{t + s)\\ds + 
Jo 



z(c(t))\ 



+ 



z{c{t + h))\ 



+ 



c(t+h)~l 

E 

i=c(t) 



U(i+l) + b(i + l) 



1*01 



Choosing i = r(rfc) and applying Gronwall's inequality to the previous in- 
equality over the interval < h < r(r/ c+ 2) — T"(rfc) gives the desired estimate. 

Since E^k) holds, \z(r)\ > \z(r k )\B~ 2 > Q^B'^M for all r £ [r fc ,r fc+2 ]. 
This observation, plus the fact that there exists -fT > such that ||6(n + 1)|| < 
, imply that e iLT °T2{r kl < f for M sufficiently large. On the other 
hand, Doob's inequality and Lemma 1 imply that 



E 



sup 



E 



U{i + 1) 



z{r k ) 



< l6m 2 E 
lQm 2 B A 

~ W k )\ 2 



E 



z(r k ) 



E[r k+2 - r k \z{r k )] 



32m 2 T B 6 32m 2 B 7 T 
< — , . , — < 



\z(r k )\ 



k-l 



Therefore 
(12) P 



sup 

rk<l<rk+2~l 



E 

i=rk 



U(i + 1) 



z [I 



>e 



-4LT ' 



z(r k ) 



< 



128m 2 T B 7 e 8LT ° 



5 2 MQ 
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Define 

£ = I sup d{4> T{r) _ T{rk) x{r k ),x{r)) <5\. 

^re[r k+1 ,r k+2 ] > 

Since r(rfc +1 ) — r(rfc) > To and x(r^) £U UV, our choice of To implies that 
x(r) € V for all r S [7*^4.1,^+2] on the event £. Inequalities (11) and (12) 
imply that P[£ ] > 1 - 128 "£™Je* LT o for M sufficiently large . D 

Lemma 4. There exists D > 0, depending only on p w , Tq, 5, a± and m 
such that 

P[E 1 (k + l)UE 2 (k) c \E 1 (k)] > 1 



for all k>l and M > sufficiently large. 

Proof. Define 

N{i) = \z{i)\ - \z(i - 1)|, D(i) = N(i) - E[N(i)\z(i - 1)] 

and 

-, r k+i 

G(k+i) = E ^(0- 



Observe that |T>(n)| < 2m. Therefore, 



^r fc +T B| Z (r fc )| 



E fl («)%-K^i 



} 



i=r fe +l 



5 2 ^ +T ^I 2 ^)I 4m 2 B 3 



where we have used the fact that TqB l \z{rk)\ < r^+i — < T$B\z{rk)\- 

2 j — - iw" 1 v — 4 J — mc* 



It follows that P[G(k + 1) < -f] < P[G(k + l) 2 > < with TJ 
16 ^T C . Since C = 1 + a l T Q /2B, it follows that 

Si(jfe + l) c n £Ji(fc) n C {G(k + 1) < -ai/2}. □ 

These three lemmas imply that there exists F > depending only on p w , 
Tq, 5, a\ and m such that 

P[E(k + 1)|^(0)] > P[E(k)\E(0)] - 
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for all k > and M > sufficiently large. Indeed, due to the fact that for 
all k > 1 E(k) equals the disjoint union of E(k + 1), E\{k + l) c n E(k), and 
E 2 {k + l) c n Si(jfe + 1) n E(k), we get 

P[E(k + l)\E(0)] 

= P[E(k)\E(0)} - P[E 2 (k + if n + 1) n £7(fc)|£(0)] 

-P[£i(£; + l) c n£(ft)|£(0)] 
>P[E(fc)|£(0)] 

- P[E 2 (k + i) c n Ei(fc) n {x(r k ) eUu V}\E(0)] 

F 



P[E ± (k + l) c n J5a(fc)|£?(0)] > P[E(k)\E(0)} 



where the first inequality follows from the inclusions E(k) C\ E\(k + 1) C 
E\{k) fi {x(rfc) € J7 U V} and -E(fc) C E 2 (k), and the second equality follows 
from Lemmas 3 and 4 with F = C + D. These inequalities remain true for 
k = 0, since -Ed(l) always holds. 
It follows that 



P 



lim £(fc) 

k— >oo 



The definition of -E(fe) implies that 

P[C]>P[£(0)](1 



m(c - 1; 



where C = {liminf^oo %i > 0} n {x n £ U U V i.o.}. On the event C, Theo- 
rem 1 implies that L({x n }) is a compact internally chain recurrent set for 
the mean limit ODE. Since L({x n }) n B{A) ^0 on the event C, a basic 
result about internally chain recurrent sets (see, e.g., [4], Corollary 5.4) im- 
plies that L({x n }) C A on the event C. Setting K = anc ^ a PPlyi n S 
Proposition 1 completes the proof of the first assertion of the theorem. To 
prove the second assertion, assume that p € Attoopf) n B(A), choose U an 
open neighborhood of p such that U C B>(A), and apply the first assertion 
of the theorem. □ 



5. Nonconvergence. In this section we show that there are two types 
of invariant sets of the mean limit ODE toward which the generalized urn 
process does not converge. The first type corresponds to a compact set where 
growth of the process is not expected, and the second type corresponds to 
a "nondegenerate" equilibrium or periodic orbit. 
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Proposition 2. Let z n be a Markov process on satisfying assump- 
tions (Al) and (A2). If K C Sk is a compact set satisfying 

(13) sup ^2p w (x)a(w) < 0, 

then P[{L(x n ) C K} n {lim n ^ 00 \z n \ = +oo}] = 0. 

Proof. Equation (13) implies that we can choose a neighborhood U of 
K, N e N and e > such that 

(14) sup E[\z n+1 \-\z n \\z n = z]<-e. 

\z\>N,z/\z\eU 

Given any I G N such that xi EU and |^| > iV, define the stopping time 

T = inf{n > l:x n (£ U or |^| < N}. 
For any n>l, we get 



0<£[|ZTA«|]=£ 



TAn 

^ - \Zi-i\ 

i=l+l 



+ E[\ Zl \ 



J2 £[|*AT|-|*(i-l)AT|]+*j|] 



i=l+l 

n 



<-eJ2 P[T>i]+E[\ Zl \}. 
i=i+i 

Taking the limit as n -> oo, we get that Ei^+i ^[^ > «] < E[\zi \]/e. The Borel- 
Cantelli lemma implies that P[T = oo] = 0. It follows that P[{L(x n ) C 
KjnOim^ool^l = +oo}] = 0. □ 

If A is a subset of R fc , then we let Span(^4) C M. k denote the vector space 
spanned by the points in A. Given a compact subset U C mt(Sfc), we say 
that the process {z n } is nondegenerate at U if for all x 6 U, 

Span{u; € Z k :p w (x) > 0} = R k . 

Recall, a periodic orbit or an equilibrium of an ODE is linearly unstable 
provided that one of its characteristic exponents is greater than zero. 

Theorem 8. Let {z n } be a generalized urn process satisfying assump- 
tions (Al) and (A2). LetlA C int(Sfc) be a linearly unstable periodic orbit or 
equilibrium for the mean limit ODE. Assume the following: 

(a) There exists (3 > 1/2 such that the functions p w are in a neighbor- 

hood oflA. 
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(b) {z n } is nondegenerate at IA. 

Then P[{L(x n ) CU)D {limint™ %i > 0}] = 0. 

Proof. Let N(U) be a neighborhood of U. The event 

I L(x n ) CU and liminf — — > ol 
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is contained in the event 



u 



E 



TV, A, 



where denotes the positive rationales and 

E N>x = {L(x n ) CU}f){\/n>N\z n \ >n\ and x n eN(U)}. 

In order to prove Theorem 8 it then suffices to prove that for iV large enough 
and A G Q+, 



P[E 



0. 



Let T n denote the sigma field generated by zq, . . . , z n and V n +i = |z n |(x n +i — 
x n ). Let F denote the vector field on Sk defined by F(x) = J2wPw(%)(w — 
xa(w)). Let {e n } denote a sequence of bounded, zero-mean i.i.d. random 
variables taking values in 



TS k = [ueR k :Y,u t = oj 1 



whose covariance matrix is nondegenerate (i.e., has rank k — 1) 
Define the sequence {x n }n>N as follows: 



X N = X N , 



(15) 



1 



(F(x n ) + V n+1 -F(x n )), 



if x n G N{U) and \z n \ > nX, 



— (F(x n ) +e n+ i), otherwise. 

^ ATI 



The processes {x n } and {x n } coincide on the event -E^A- On the other 
hand, 



lim dist(x n ,U) = 







in view of the following theorem whose proof is an easy adaptation of [13], 
Theorem 2. 
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Theorem 9. Let {J- n } n ez+ denote a nondecreasing sequence of sub-a -algeb- 
ras of J- and (x n ) a sequence of adapted random variables given by 

(16) %n+l ~ %n — Pn{F(x n ) + U n +1 + b n +l), 

where F is a C 1+/3 vector field, with 1/2 < ft < 1, {U n }, {b n } and {/?„} are 
adapted random variables. 

We define V n+ i = F(x n ) + U n+ i + b n+1 . 

Assume the following: 

(i) 3K > 0, Vn € Z + \\U n \\ < K and E(U n+1 \T n ) = 0. 

(ii) There exist a, b > and a deterministic sequence {7 n } of nonnegative 
numbers having infinitely positive terms, such that\/n € Z+, aj n < (3 n < 
bin- 

(iii) £6?<+oo. 

(iv) U C Int(5fc) is a linearly unstable periodic orbit or equilibrium for F . 

(v) There exist a neighborhood N(Ll) oflA and c> such that, for all unit 
vectors veM. m , E(\(V n+1 ,v)\\f n ) > cl {ineN{u)} . 

Then P[L(x n ) C U] = 0. 
Let 

A n = {x n G N(U) and \z n \ > n\}. 
Using the notation of Lemma 1 set 

Un+i = Un+i, b n+ i = b n+ \, [j n = \/\z n \ on A n 

and 

U n+ i=e n+ i, 6 n+ i = 0, (3 n = l/\n on A c n . 

Then, by Lemma 1, the process {x n } defined by (15) verifies recursion (16) 
and assertions (i)-(iv) of Theorem 9 are satisfied. 

It remains to verify assertion (v). Let -B(l) ={»£ <Sfc : IMI = !}■ Let 
d : S k x B(l) -> M+ and G 2 :UxS k x B(l) -> M+ be the functions defined 
by 

G 1 (x,v) = E(\(F(x)+e n ,v)\) 

and 

G 2 (x,f,?;) =^|(F(f) -F(x) + Q x (e),v)\p e (x), 

e 

where Q x denote the projection operator Q x : Span{x} © TS k — > TS k - 
Then it is not hard to verify that 

E(\(V n+u v) | \T n ) = Gi (x n ,v)l A o + G 2 (x, x n , v)l An + 0(l/n). 

By continuity of G%, G 2 compactness of the sets U,S k ,B(l) and assump- 
tion (A2), there exists b > and a neighborhood N{U) such that Gi(x,v) > 
6, G 2 (x,x,v) > 6 for all x 6 N(U), x £ S k and u 6 -B(l) 

Assumption (v) is thus verified for n> N and N large enough. □ 



EVOLUTIONARY URN MODELS 



23 



6. Nondegenerate processes with gradient-like mean limit ODEs. Using 
the results from the previous two sections, we can prove the following result. 

Theorem 10. Let z n be a generalized urn process satisfying assump- 
tions (Al) and (A2), p w are C 1+/3 for some j3 > 1/2, z n is nondegener- 
ate on Sk, Attoo(A) = Sk and the chain recurrent set for the mean limit 
ODE consists of hyperbolic equilibria q that satisfy J2 w Pw(Q) a ( w ) 7^ 0. Then 
P[liminf n _ >oo > 0] if and only if there exists a linearly stable equilibrium 
q such that ^2 w p w {<l)oi{w) > 0. Furthermore, P[{L(x n ) = q}] > for any 
linearly stable equilibrium q satisfying J2wPw(q)ct(w) > 0, and P[{L(x n ) = 
q}] = for any equilibrium q which is linearly unstable. 

Proof. Define Q = {limin^^oo \z n \/n > 0}. Since z n is nondegenerate, 
Theorems 1 and 8 imply that L(x n ) is contained almost surely in the set of 
linearly stable equilibria on the event Q. If J2 w Pw(l) a ( w ) < f° r au linearly 
stable equilibria q, then Proposition 2 implies P[G] =0. Alternatively, if q is 
a linearly stable equilibrium and Yj W Pw{o)oi{w) > 0, then Theorem 7 implies 
P[Q fl {L{x n ) = q}] > 0. Finally, if q is an equilibrium which is linearly un- 
stable, then Proposition 2 implies P[L(x n ) = q] = if J2 w Pw(Q) a ( w ) < an d 
Proposition 1 and Theorem 8 imply P[L(x n ) = q] = if YlwPw{.<l)&{w) > 0. 
□ 

As an application of this result, we consider additive fertility-selection 
processes z n with mutation, where g(ij,rs) = E[G n (ij,rs)] is the expected 
number of progeny produced by a mating between genotypes AiAj and 
A r A s , and fJ,(ij, rs) is the probability genotype AiAj mutates to genotype A r A s . 

Corollary 1. Let z n be an additive fertility-selection process with mu- 
tation. Suppose: 

• \/i,j,r,s G {l,...,k}, n{rs,ij) is strictly positive, and sufficiently small 
when {r, s} / {i,j}- 

• \/i,j,r, s £ {l,...,fc} and such that the fertility- selection equation (8) 
without mutation has hyperbolic equilibria q satisfying ^2 w p w {<l)ot{w) ^ 0, 
P[G 1 (ij,rs)>3]>0. 

Then -Pfliminfj^oo > 0], if and only if there exists a linearly stable equi- 
librium q such that J2wPw(q)o(w) > 0. Furthermore, P[{L({x n }) = q}] > 
for any linearly stable equilibrium q satisfying J2wPw(q)o:(w) > 0, and P[{L{x n } = 
q}] = for any equilibrium q which is linearly unstable. 



24 



M. BENAIM, S. J. SCHREIBER AND P. TARRES 



Proof. Since the mean limit ODE corresponding to the fertility-selection 
process without mutation is gradient-like and has only hyperbolic equilibria, 
the chain-recurrent set for this mean limit ODE equals the set of equilibria. 
Consequently, the mean limit ODE corresponding to the fertility-selection 
process with sufficiently small mutation rates also has a chain-recurrent 
set consisting only hyperbolic equilibria. Due to the fact that all mutation 
rates are positive, this process is nondegenerate on the entire simplex. Since 
P[G\(ij,rs) > 3] > for all 1 < i,j,r,s < k, Att oa (X) is the entire simplex. 
Applying Theorem 10 completes the proof. □ 
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